{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "99f41d2d",
   "metadata": {},
   "source": [
    "![LOGO](../../../img/MODIN_ver2_hrz.png)\n",
    "\n",
    "<center><h2>Scale your pandas workflows by changing one line of code</h2>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdda1c9c",
   "metadata": {},
   "source": [
    "# Exercise 4: Experimental Features\n",
    "\n",
    "**GOAL**: Explore some of the experimental features being added to Modin."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b487c51",
   "metadata": {},
   "source": [
    "### Concept for exercise: Progress Bar\n",
    "\n",
    "\n",
    "Sometimes when running long functions on DataFrames, it can be hard to tell how much progress has been made, as well as how much longer the function will run. A progress bar allows users to see the estimated progress and completion time of each line they run, in environments such as a shell or Jupyter notebook.\n",
    "\n",
    "To enable Modin's Progress Bar, add the following lines of code after importing `modin.pandas`:\n",
    "```python\n",
    "from modin.config import ProgressBar\n",
    "ProgressBar.enable()\n",
    "```\n",
    "\n",
    "In this exercise, we'll see how the progress bar can improve our experience running dataframe queries!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f95d4874",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import modin.pandas as pd\n",
    "import numpy as np\n",
    "from modin.config import ProgressBar\n",
    "ProgressBar.enable()\n",
    "\n",
    "frame_data = np.random.randint(0, 100, size=(2**18, 2**8))\n",
    "df = pd.DataFrame(frame_data).add_prefix(\"col\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6905bc6b",
   "metadata": {},
   "source": [
    "On longer functions, its nice to be able to see an estimation of how much longer things will take!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "236ec8e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.applymap(lambda x: ~x)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7bf87a5",
   "metadata": {},
   "source": [
    "### Concept for exercise: Spreadsheet\n",
    "\n",
    "For those who have worked with Excel, the Spreadsheet API will definitely feel familiar! The Spreadsheet API is a Jupyter notebook widget that allows us to interact with Modin DataFrames in a spreadsheet-like fashion while taking advantage of the underlying capabilities of Modin. The widget makes it quick and easy to explore, sort, filter, and edit data as well as export the changes as reproducible code.\n",
    "\n",
    "Let's look back at a subset of the 2015 NYC Taxi Data from Exercise 2, and see how the Spreadsheet API can make it easy to play with the data!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5d5c4a3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "!jupyter nbextension enable --py --sys-prefix modin_spreadsheet\n",
    "ProgressBar.disable()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc8d5903",
   "metadata": {},
   "outputs": [],
   "source": [
    "import modin.experimental.spreadsheet as mss\n",
    "\n",
    "s3_path = \"s3://dask-data/nyc-taxi/2015/yellow_tripdata_2015-01.csv\"\n",
    "modin_df = pd.read_csv(s3_path, parse_dates=[\"tpep_pickup_datetime\", \"tpep_dropoff_datetime\"], quoting=3, nrows=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "145e7bbe",
   "metadata": {},
   "outputs": [],
   "source": [
    "spreadsheet = mss.from_dataframe(modin_df)\n",
    "spreadsheet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c18b7f2",
   "metadata": {},
   "source": [
    "### Thank you for participating!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
